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SUMMARY 

The  study  documented  in  this  paper  was  a three-month  effort  supported 

by  the  Precision  Laser  Designator  Program  Office  within  the  USA  MICOM.  The 

stuc|y  was  designed  to  establish  the  validity  and  nature  of  a relationship 

* 

between  the  cost  of  producing  prototype  units  and  the  cost  of  producing  the 
units  In  the  investment  phase  where  only  costs  recurring  with  the  units  was 
considered.  A secondary  objective  of  this  analysis  involved  the  assessment 
of  the  most  appropriate  cost  improvement  theory  to  employ  in  developing  the 
theoretical  first  unit  cost  of  an  item. 

The  methodology  employed  considered  several  prominent  hypotheses  on 
the  form  and  variable  content  of  the  relationship  between  prototype  and 
production  units  and  also  involved  the  identification  of  two  methods  for 
assessing  the  validity  of  the  two  improvement  curve  theories.  Testing  of 
these  hypotheses  on  data  for  16  electronics  subsystems  for  various  applica- 
tions produced  several  good  models  of  the  prototype/production  relationship. 

The  figure  shows  the  results  for  one  of  the  models  assuming  log  linear 
cumulative  average  costs  which  fit  the  data  very  well  with  little  scatter 
evident.  This  model  represents  a simple  proportional  relationship  between 
Tj  • the  theoretical  first  production  unit  cost,  and  Tjp  , the  theoretical 
first  unit  prototype  cost,  derived  based  on  the  improvement  rate  observed 
In  the  production  lot  data  and  the  average  prototype  cost.  The  particular 
hypothesis  shown  In  this  figure,  called  the  sequential  hypothesis,  assumes 
that  the  production  quantities  add  to  and  continue  in  sequence  the  proto- 
type quantity.  The  alternative  to  this  theory,  called  the  disjoint  theory, 
assumes  the  improvement  curve  for  production  begins  again  at  unit  number  one. 
Ne  were  unable  to  determine  statistically  which  of  these  two  theories  was 
better  because  both  produced  good  results.  However,  on  the  basis  of  the 
statistical  results,  we  did  conclude  that  prototype  cost  was  a good 
estimator  of  production  costs. 


* 

* t 

Similarly,  we  were  unable  to  establish  the  superior  inprovement  rate 
theory.  However,  it  was  established  that  for  typical  first  lot  production 
quantities,  log  linear  fits  of  production  lot  data  under  either  theory  are 
virtually  Identical.  In  addition.  It  was  shown  that  there  Is  some  impact 
on  the  prototype/production  cost  relationship  where  one  theory  Is  correct 

and  the  other  Is  assumed  and  where  prototype  quantities  are  small. 

/ 

The  analysis  also  revealed  (1)  no  distinguishable  difference  between 
the  relationship  for  missiles  and  other  electronics  units,  and  (2)  that 
cost  Improvement  occurs  In  the  production  of  prototypes  which  should  be 
taken  into  account  in  estimating  production  costs. 


1.0 


BACKGROUND  AND  OBJECTIVE  OF  STUDY 

1.1  BACKGROUND 

In  general,  it  has  been  an  established  practice  in  the  cost  analysis 
community  to  apply  a cost-to-cost  factor  to  the  cost  of  producing  prototype 
hardware  to  obtain  an  estimate  of  manufacturing  this  hardware  in  the  produc- 
tion or  investment  phase.  The  use  of  such  a methodology  in  making  cost  esti- 
mates involve  (1)  an  assumption  that  a relationship  between  prototype  produc- 
tion and  investment  phase  production  costs  exists,  and  (2)  the  development  of 
a factor  based  usually  on  limited  information.  There  is  good  justification 
for  the  assumption.  Essentially,  the  same  performance  characteristics  apply 
for  each  model  and  often  the  models  contain  many  of  the  same  component  parts. 
That  this  approach  is  used  rather  widely,  alone,  is  one  justification  for  its 
validity.  However,  to  our  knowledge,  there  has  been  no  careful  statistical 
verification  of  such  a relationship. 

1.2  STUDY  OBJECTIVE 

In  light  of  the  above  facts,  Tecolote  Research,  Inc.  (TRI)  was  asked 
by  the  Laser  Designator  Program  Office  (USA  MICOM)  to  investigate  the  quan- 
titative validity  of  this  hypothesis  for  electronic  programs--especially  as 
they  pertain  to  the  requirements  of  the  Ground  and  Airborne  Laser  Locator 
Designators  (GLLD  and  ALLO,  respectively).  Specifically,  two  principal  tasks 
were  addressed  in  this  analysis: 

• Examine  the  validity  of  a relationship  between  the  cumulative 
average  cost  (independent  variable,  Cp  ) of  engineering  develop- 
ment phase  (prototype  production)  hardware  to  the  first  unit  cost 
of  producing  Investment  phase  hardware  (dependent  variable,  Tj  ), 
and  present  substantiation  of  the  position  on  validity  including 
any  resulting  relationship. 

e Examine  the  validity  of  two  cost  improvement  theories,  log-linear 
cumulative  average  and  log  linear  unit  in  determining  the  theoreti- 
cal first  unit  cost  of  production  (Tj)  used  in  the  above  relation- 
ship. 
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The  remaining  sections  of  this  document  addressing  these  objectives 
are  organized  in  the  following  way:  Section  2.0  addresses  the  methodology 
employed  in  the  study  and  develops  the  rationale  for  the  formulation  of  the 
various  cost  estimating  relationships.  Section  3.0  outlines  the  data  base 
used  for  this  study  including  a discussion  on  work  breakdown  structure  (WBS) 
detail,  normalizations,  and  data  sources.  Section  4.0,  Hypothesis  Testing, 
presents  the  results  of  the  study  both  from  the  standpoint  of  the  principal 
tasks  outlined  earlier  as  well  as  other  observations  and  results  that  were 
revealed  in  the  course  of  the  analysis. 


2.0  HYPOTHESIS  DEVELOPMENT 


The  goals  of  the  hypothesis  development  task  are  twofold:  (1)  to 
develop  a model  which  relates  a variable  depicting  the  cost  of  prototype 
units  to  a measure  of  investment  phase  production  costs  (theoretical  first 
unit  cost,  Tj  );  and  (2)  to  develop  a method  for  the  determination  of  the 
most  appropriate  cost  improvement  theory.  Refinements  of  the  basic  factor 
model  in  terms  of  other  cost  impacts  are  also  a consideration  in  the 
development  of  the  hypothesis. 

The  major  issues  that  need  resolution  related  to  these  goals  are: 

(1)  Is  the  production  cost  improvement  curve  a continuation  of  the 
prototype  cost-quantity  curve  with  the  exception  of  a displace- 
ment in  the  curve  when  production  begins. 

(2)  Does  the  quantity  of  prototypes  built  affect  the  relationship 
between  production  and  prototype  costs? 

(3)  Is  log-linear  cumulative  average  cost  theory  more  appropriate 
than  log-linear  unit  cost  theory  in  depicting  cost  quantity 
relationships? 

Concerning  the  first  of  these  issues,  there  are  two  prominent  theories 
and  they  differ  only  with  respect  to  production  quantity.  One  theory  which 
we  call  "disjoint"  (Figure  2.1)  consideres  the  first  production  unit  as  unit 
number  one,  while  the  other  theory  called  "sequential"  (Figure  2.2)  considers 
the  first  production  unit  as  unit  number  (Qp+1)  , where  Qp  is  the  number 
of  prototype  units.  The  result  of  the  sequential  assumption  is  a differing 
cost  impact  in  production  (cost  reductions  over  the  early  production  units 
Is  smaller),  and  Tj  , the  theoretical  first  unit  cost,  is  measured  at  a 
different  point  (the  first  prototype  unit).  As  is  evident  Tj/Tjp  is  greater 
In  the  sequential  theory  than  in  the  disjoint  theory.  Here  the  improvement 

^Appendix  B discusses  an  alternative  formulation  of  the  production/prototype 
cost  relationship  wherein  a theoretical  first  unit  cost  and  improvement 
rate  are  not  derived  for  each  program.  Instead,  the  complete  cost-quantity 
population  is  estimated  in  a single  regression  model.  This  is  an  uncon- 
ventional approach  which  was  not  pursued  at  length,  but  the  results  are 
included  for  the  interested  analyst. 


PROTOTYPE 


PRODUCTION 


FULL  SCALE 


PRODUCTION 


FIGURE  2.1 

THE  DISJOINT  HYPOTHESIS 


PROTOTYPE 


RODUCTION 


TILL  SCALE 


RODUCTION 


FIGURE  2.2 

THE  SEQUENTIAL  HYPOTHESIS 


rate  derived  from  the  production  lot  data  is  applied  to  the  prototype  cost 
data  to  determine  Tjp  . Tj  in  the  sequential  case  is  determined  .j t an 
earlier  quantity  (the  first  prototype  unit).  The  effect  of  adding  prototype 
quantities  to  the  production  lot  data  is  to  steepen  the  improvement  curve 
fit  to  the  lot  data  as  the  lot  data  at  small  quantities  is  shifted  farther 
to  the  right  by  the  addition  of  prototype  units  than  that  data  at  larger 
quantities.  The  selection  of  the  best  hypothesis  among  these  two  will 
depend  on  which  produces  the  smallest  estimating  error. 

Initially,  we  intended  to  examine  only  the  relationship  between  the 
average  prototype  cost  (Cp)  and  Tj  ; however,  we  feel  that  variations 
in  Cp  because  of  differing  quantity  values  is  significant  and  that  a 
better  measure  of  prototype  cost  is  the  theoretical  first  prototype  unit 
cost  calculated  assuming  that  production  improvement  rates  are  the  same  as 
would  be  observed  in  the  prototype  phase.  This  presumes  that  any  error  in 
such  an  extrapolation  of  improvement  rates  is  less  than  the  quantity  varia- 
tion alluded  to.  We  will  test  this  hypothesis  by  examining  Tj  as  a func- 
tion of  and  Cp  separately. 

The  quantity  of  prototypes  (Qp)  should  also  be  considered  as  an  in- 
dependent variable  for  several  reasons.  The  incorporation  of  Qp  could 
indicate  the  use  of  an  improper  improvement  rate  in  the  prototype  phase. 

If  there  were  no  cost  improvement  in  the  prototype  phase,  we  would  have 
overstated  Tjp  and  Qp  would  compensate  by  having  a negative  coefficient 
under  each  hypothesis.  Alternatively,  a negative  coefficient  in  the  dis- 
joint hypothesis  might  suggest  that  there  is  learning  transfer  between  the 
prototype  and  investment  phase;  i.e.,  larger  prototype  quantities  lead  to 

ick 

a lower  Tj  , an  implicit  feature  of  the  sequential  model.  Such  a result 
might  verify  the  sequential  hypothesis.  A prototype  quantity  variable 
should  be  employed  along  with  Cp  to  assess  the  significance  of  cost  im- 
provement. Economies  of  scale  in  production  can  be  tested  by  examination 
of  the  non-linearity  of  Tj  with  Tlp  . 

* 

Tj  is  estimated  assuming  that  the  prototype  buy  is  a missing  lot  and 

this  is  then  compared  with  Tlp  . 

**Tj  in  the  disjoint  model  is  unit  number  Qp+1  in  the  sequential  model. 


8 


The  remaining  issue  of  importance  is  the  appropriate  cost  improve- 
ment theory  to  apply  to  determine  first  unit  cost  Tj  . The  obvious  crite- 
rion here  is  which  theory  fits  the  production  lot  data  best.  Unfortunately, 
the  difference  between  each  theory  is  apparent  only  at  quantities  that  are 
small  relative  to  typical  production  lot  quantities.  This  i§  true  because 
the  cumulative  average  unit  cost  curve  asymptomatically  parallels  a log 
linear  unit  cost  curve  so  that  a unit  cost  fit  is  indistinguishable  from  a 
cumulative  average  unit  cost  curve  as  is  shown  in  Figure  2.3.  The  error 
in  assuming  log  linear  unit  cost  if  cumulative  average  theory  is  true  is 
shown  in  Figure  2..4 for  an  85J  improvement  rate.  This  error  is  smaller  for 
lower  improvement  rates.  The  level  of  error  after  a few  units  is  small  with 
respect  to  typical  cost  estimating  uncertainty. 

Because  prototype  quantities  are  smaller  than  their  typical  production  lot 
buys  suggest  that  prototype  data  might  provide  a better  indicator  of  which 
theory  is  appropriate.  This  position  may  be  illuminated  by  reference  to 
Figure  2.5  where  we  contrast  the  results  of  the  factor  model  fits  for  large 
and  small  quantity  prototype  programs.  The  application  of  an  incorrect  im- 
provement theory  where  only  a few  prototypes  are  built  leads  to  a bias  in 
the  determination  of  the  factor  relating  production  to  prototype  cost.  This 
figure  involves  the  following  assumptions:  (1)  that  the  disjoint  hypothesis 
applies;  (2)  that  log  linear  cumulative  average  cost  theory  holds  true; 

(3)  that  the  production  and  prototype  improvement  rates  are  equal;  and 

(4)  that  is  equal  to  T^p  , i.e.,  the  factor  Tj/Tjp  equals  one. 
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GRAPHICAL  EXPOSITION 
CAC  (PROTOTYPE  S FULL-SCALE) 


/ REAL  UNIT  COST  (PROTOTYPE) 


LIRE  2.5 

lp  OF  THE  APPLICATION 
IMPROVEMENT  THEORY 
PROTOTYPE  QUANTITIES 


If  log-linear  unit  theory  is  applied  to  the  production  data,  the 
resulting  fit,  given  that  first  lot  buy  is  not  small,  will  be  essentially 
the  same  as  the  real  unit  cost  curve  under  cumulative  average  theory  at 
these  quantities.  Similarly,  the  application  of  unit  theory  to  the  proto- 
type data  will  produce  an  approximation  of  the  real  unit  cosl  curve  where 

★ 

prototypes  are  not  small  as  indicated  by  Qpi  . Should  the  prototype  quan- 
tity be  small,  however,  (0p2^  » then  the  rea^  unit  cost  wou^  lie  above 
the  investment  data  fit  and  the  calculated  first  unit  prototype  cost 
(Tip/gj)  would  be  greater  than  Tj  , producing  a factor  that  is  less  than 
one.  This  indicates  that  the  application  of  the  incorrect  improvement 
theory  biases  the  real  factor  and  the  degree  of  bias  depends  on  the  quan- 
tity of  prototypes  built.  This,  in  turn,  suggests  that  the  incorrest 
theory  will  produce  poorer  fit  statistics  where  a cross  section  of  programs 
are  examined  and  one  theory  tends  to  apply.  Poorer  factor  model  fits  in 
one  theory  given  small  prototype  quantities  may  then  present  a basis  for 
selecting  the  most  appropriate  theory. 


* 

Because  the  factor  is  equal  to  one,  the  production  and  prototype  improve- 
ment curves  under  unit  theory  should  approximately  overlap  given  a large 
prototype  quantity. 

** 

Where  log  linear  unit  theory  is  true  and  cumulative  average  theory  is 
applied,  there  will  be  a similar  bias  in  the  opposite  direction. 
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3.0  DATA  REDUCTION  AND  NORMALIZATION 

The  analysis  of  the  relationship  between  prototype  and  production 
costs  presents  fairly  stringent  requirements  on  the  data  to  be  employed 
and  substantially  limits  the  number  of  usable  data  points.  Of  course, 
both  prototype  and  investment  phase  data  is  required  and  this  data  must 
exclude  all  of  the  non-recurring  items  such  as  design  engineering  in  the 
prototype  phase.  Only  electronics  items  were  considered.  Specifically 
Included  in  the  individual  item  costs  are  the  recurring  costs  for  the 
items  identified  in  Form  DD-1558-1;  i.e.,  recurring  engineering  tooling, 
quality  control,  manufacturing  material,  and  purchased  equipment.  Ex- 
cluded are  such  system  level  items  as  program  management,  data  and  system 
engineering.  The  data  set  was  also  reduced  by  the  exclusion  of  programs 
where  prototype  phase  data  of  appropriate  detail  was  available,  but  pro- 
duction data  was  available  only  in  the  form  of  an  estimate.  Probably 
the  most  restrictive  constraint  was  the  need  for  recurring  costs  in  the 
prototype  phase. 

We  expected  that  the  relationship  between  prototype  and  investment 
units  would  vary  with  the  type  of  equipment.  To  verify  this,  we  attempted 
to  Incorporate  a variety  of  electronics  systems  in  the  data  set.  Our  best 
source  of  data  initially  was  in  the  tactical  missile  area,  so  we  focussed 
our  effort  on  adding  avionics  and  ground  equipment  to  this  base.  We  also 
disaggregated  the  cost  data  as  much  as  was  possible  in  the  equipment  (WBS) 
dimension  in  order  to  broaden  the  sample  and  to  provide  visibility  for 
assessing  factor  variations  with  equipment  type.  Usually,  such  a break- 
down is  limited  to  Level  3 (the  subsystem  level)  of  a WBS. 

The  cost  data  employed  in  this  report  are  summarized  in  Tables  3.1 
through  3.3.  All  of  the  data  shown  in  these  tables  have  been  normalized 
for  cost,  quantity,  and  fiscal  escalation  using  a TRI  computer  routine 
called  C0STN0R.  All  dollar  figures  are  presented  In  constant  fiscal  year 
1974  dollars  and  were  escalated  to  this  by  using  the  OSD  (Comptroller) 
procurement  escalation  index  shown  in  Table  3.4.  In  all  cases,  the  data 
presented  Include  general  and  administrative  expense,  but  do  not  include 
contractor  fee. 
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TABLE  3.2  DATA  SET  FOR  THE  DISJOINT  HYPOTHESIS' 

ALL  COSTS  IN  THOUSANDS  OF  FY  1974  DOLLARS 
LOG  LINEAR  CUMULATIVE  AVERAGE  COST  IMPROVEMENT  CURVE  THEORY  ASSUMED 
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TABLE  3.3  DATA  SET  FOR  THE  DISJOINT  HYPOTHESIS 
ALL  COSTS  IN  THOUSANDS  OF  FY  1974  DOLLARS 
LOG  LINEAR  UNIT  COST  IMPROVEMENT  CURVE  THEORY  ASSUMED 
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3.1  PRODUCTION  LOT  DATA  ANALYSIS 

The  availability  of  a TRI  developed  computer  program  called  C0STN0R 
greatly  expedited  much  of  the  tedious  (and,  therefore,  error-prone)  and 
time-consuming  cost  data  normalization  effort  undertaken  in  estimating  pro- 
duction first  unit  costs  for  both  quantity  theories.  As  used  in  this  study, 
the  program  was  utilized  for  determining  values  of  the  cost  improvement 
curve  parameters  when  lot  cost  and  lot  quantity  data  were  given  for  two 
(or  more)  production  lots.  The  data  need  not  be  sequential  and  need  not 
include  the  first  production  lot.  This  feature  allowed  the  determination 
of  Tj  under  the  discontinuous  hypothesis  by  considering  the  prototype 
lot  as  a missing  lot  of  quantity,  Qp  . 


The  C0STN0R  routine  employs  an  iterative  procedure  to  establish  the 
theoretical  first  unit  cost  and  improvement  rate  under  either  log  linear 
cumulative  average  or  log  linear  unit  theory.  It  operates  on  lot  average 
cost  data  so  that  no  smoothing  of  data  occurs,  thereby  retaining  all  the 
information  content  of  the  original  data.  This  also  permits  the  determina- 
tion of  cumulative  average  curves  where  some  lot  data  is  missing. 

Appendix  A describes  this  computer  program  in  greater  detail. 
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3.2  PROGRAM  DATA  DESCRIPTIONS 


I 


I 


The  following  subsection  briefly  addresses  each  of  the  data  elements 
used  in  this  analysis. 

3.2.1  Phoenix  Missile  - Guidance  and  Control  r 

The  Phoenix  missile  cost  data  were  obtained  directly  from  the  account- 
ing records  of  the  Hughes  Aircraft  Co.  production  facility  in  Tucson,  Arizona, 
and  the  Missile  Division  at  Canoga  Park.  The  data  for  production  during  fis- 
cal years  *73,  '74,  and  '75  are  actual  recurring  costs  incurred  based  on  per- 

★ 

sonnel  time  card  entries  and  material  costs.  The  FY'76  data  are  actuals 
plus  purchase  order  commitments  as  of  6 June  1975.  Quantity  information  for 
FY'76  is  the  number  of  "equivalent  units".  The  accounting  records  are  main- 
tained separately  for  each  fiscal  year  '73  through  '74. 


The  production  cost  accounting  records  are  very  detailed,  providing 
fabrication,  assembly,  inspection  and  test,  and  material  cost  information 
for  each  WBS  element.  These  detailed  accounting  records  were  aggregated 
and  summarized  by  WBS  elements.  Fabrication  and  Assembly  activities  were 
judged  to  be  most  nearly  comparable  with  CIR/CCDR  descriptions  of  manufac- 
turing. Similarly,  Inspection  and  Test  were  judged  to  be  comparable  to 
quality  assurance.  It  was  not  possible,  however,  to  relate  the  engineering 
costs  to  WBS  elements,  nor  was  it  possible  to  clearly  identify  the  recurring 
tooling  costs.  CIR's  for  the  Phoenix  were  examined  to  verify  these  observa- 
tions. All  of  the  1558-1  CIR  forms  available  at  this  time  (through  12/74) 
confirm  that  recurring  tooling  costs  are  not  identifiable.  It  was  explained 
that  recurring  tooling  costs  are  accrued  as  incurred  into  the  activity  ac- 
counts described  above  (e.g.,  fabrication,  assembly,  etc.).  Factory  sustain- 
ing and  support  costs  could  not  be  related  to  WBS  elements  and  were,  there- 
fore, aggregated  and  allocated  to  the  hardware  elements. 


*The  fiscal  years  indicated  here  represent  the  approximate  data  at  the  mid- 
point of  the  actual  delivery  schedule. 


The  RDT4E  phase  cost  data  was  obtained  from  the  accounting  records  for 
three  separate  contracts:  (1)  the  initial  RDT4E  program,  (2)  the  Test  Proto- 
type Missile  program,  and  (3)  the  Separation  and  Test  Missile  program.  Infor- 
mation was  available  to  the  subsystem  level  on  manufacturing  and  test  costs. 
Production  (factory)  support  costs  were  not  available  to  this  level,  however, 
and  were  allocated  to  the  subsystems  on  the  basis  of  particular  subsystems 
cost.  The  average  prototype  cost  (Cp)  in  Table  3.1  - 3.3  is  the  average 
for  all  three  contracts. 

3.2.2  Maverick  Missile  - Guidance  and  Control 

Maverick  missile  production  cost  data  were  obtained  from  the  System 
Program  Office  (SPO)  at  Wright-Patterson  Air  Force  Base,  Dayton,  Ohio.  The 
data  represent  the  results  of  a detailed  analysis  of  contractor  records  con- 
ducted by  the  SPO  over  a period  of  several  weeks  at  the  Hughes  Aircraft  Co. 
production  facility  in  Tucson,  Arizona.  Adjustments  were  made  for  material 
burden,  material  allowances,  abnormal  economic  escalation,  and  contract  in- 
centives. The  allocations  and  adjustments  were  performed  by  SPO  personnel 
with  assistance  and  general  guidance  from  HAC. 

There  were  data  from  four  full  scale  production  contracts  (Options  A, 

B,  C,  D)  and  one  pilot  production  contract  used  to  fit  the  cost  improvement 
curve  for  the  Maverick  Missile. 

The  prototype  data  were  obtained  from  the  Hughes  Aircraft  Co.,  Canoga 
Park  facility.  These  data  did  not  provide  the  level  of  detail  available 
for  the  production  history,  hence,  production  cost  data  were  aggregated  to 
a level  comparable  with  the  prototype  information.  Prototype  fabrication 
occurred  in  Canoga  Park,  California,  (an  R&D  facility)  and  full  scale 
production  in  the  Tucson,  Arizona,  production  plant.  The  prototypes  were 
built  as  part  of  the  single  DDT4E  program  which  also  involved  the  fabrica- 
tion of  the  pilot  production  missiles  or  Tucson  "G"  missiles  built  at  the 
Hughes  Tucson  facility.  The  detailed  accounting  records  for  this  program 
allowed  distinguishing  between  the  costs  for  each,  which  cost  performance 
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and  CIR  reports  did  not  permit.  The  prototype  cost  detail,  however,  did 
not  allow  the  separation  of  guidance  and  control  sections  except  for  the 
hydraulic  actuation  system  which  was  not  of  concern  in  this  study.  Proto- 
type and  pilot  production  model  quantities  were  obtained  from  Maverick  SPO. 

f 

3.2.3  Standard  Missile  - Guidance  and  Control 

Standard  missile  production  cost  data  were  obtained  from  the  General 
Dynamics  Corporation  production  facility  in  Pomona,  California.  The  data 
were  extracted  from  ledgers  in  their  industrial  accounting  office  and  rep- 
resent actual  costs  incurred.  The  contractor  maintains  a contract  WBS  to 
which  he  assigns  accounting  work  order  (AWO)  numbers.  While  the  WBS  struc- 
ture appeared  to  be  quite  detailed,  it  provided  very  limited  hardware  visi- 

★ 

bility  below  the  missile  section  level;  e.g.,  guidance  section. 

Prototype  cost  data  (Contract  No.  N0W65-01 19i ) was  also  extracted 
from  General  Dynamics  accounting  data.  This  contract  also  funded  the 
production  of  the  100  pilot  line  missiles.  The  accounting  record  detail 
allowed  the  separation  of  costs  for  guidance,  control,  and  ordnance  sec- 
tions.  The  average  costs  for  the  standard  prototype  sections  may  be 
slightly  overstated  since  the  prototype  fabrication  assembly  and  checkout 
task  encompassed  some  completion  of  the  design  task  of  the  prior  contract. 

3.2.4  AWG-9  Fire  Control  Radar 

The  cost  data  for  the  AWG-9  system  was  obtained  from  CIR  reports  for 
both  RDT&E  and  investment  phase  equipment.  Production  lot  data  consisted 
of  production  buys  for  FY ' 7 1 through  FY'75.  The  CIR's  provided  this  infor- 
mation at  the  subsystem  level,  i.e.,  radar,  computer,  etc.  Indicated  CIR 
quantities  contained  equivalent  spares  where  the  costs  for  spares  were 

separately  identified  necessitating  quantity  adjustments. 

_ 

In  the  course  of  the  data  analysis,  it  was  learned  that  a pilot  production 
program  for  100  missiles  had  preceded  the  first  full  scale  production  con- 
tract. While  cost  data  was  not  available  for  the  pilot  production  effort, 
the  improvement  curve  analysis  allows  for  these  100  units  by  treating  them 
as  missing  lot  data. 

**0rdnance  section  cost  data  was  not  analyzed  in  this  study  because  of  the 
many  participants  (contractor  and  government)  in  both  RDTAE  and  investment 
phase  and  the  corresponding  problem  of  complete  cost  data. 
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The  prototype  cost  information  was  obtained  from  Contract 
No.  N00019-70-C-0207  wherein  three  prototype  models  (XN-3)  and  four  pilot 
models  (YN-3)  were  produced.  These  three  prototypes  were  the  last  of  a 
series  of  25  prototype  models  built  in  the  AWG-9  RDT&E  phase.  Recurring 
costs  for  prototypes  of  earlier  contracts  could  not  be  established  by 
the  contractor.  It  was  necessary  to  estimate  recurring  engineering  for 
the  pilot  production  units  in  order  to  establish  prototype  recurring  costs 
in  the  -0207  contract.  This  cost  was  estimated  as  equal  to  the  average 
recurring  engineering  fraction  observed  in  the  following  four  production 
lots  where  little  variance  in  this  fraction  was  noted. 

3.2.5  Improved  HAWK  - Ground  Fire  Control 

Cost  and  quantity  data  for  the  Improved  Hawk  (I/H)  acquisition  radar 
(AR),  data  processor  (ICC),  high-powered  illuminator  (HPI),  and  battery 
control  center  (BCC)  were  obtained  from  CEIS  files  at  USA  MI  COM.  The  in- 
formation for  both  prototype  and  full-scale  production  was  retrieved 
directly  (with  appropriate  fiscal  escalation)  from  government  forms  DD-1737 
and  DD-1177,  the  reporting  forms  required  to  be  completed  by  the  prime  con- 
tractor (Raytheon,  Andover  Plant)  for  the  single  prototype  production  con- 
tract (three  units)  and  the  three  full-scale  production  contracts.  Complete 
and  reliable  data  was  not  available  on  the  fourth  and  last  full-scale 
production  buy. 

In  all  cases,  cost  data  represents  recurring  costs  for  engineering, 
tooling,  quality  control,  and  manufacturing  and  material.  A G&A  rate  of 
11.3%  was  shown  to  be  representative  by  USA  MICOM  personnel  and  was  applied 
to  the  direct  charges. 


\ 

‘ 
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3.2.6  Doppler  Navigation  Systems 

Costs  for  these  avionics  units  were  provided  by  the  Canadian  Marconi 
Corporation  based  on  their  accounting  records  and  corresponding  to  the  cost 
elements  of  Army  Regulation  37-18.  No  production  lot  data  was  available; 
the  contractor  indicated  that  a 90%  learning  rate  was  consistent  with  their 
experience.  These  units  were  all  developed  in  the  mid  to  late  1960's  with 
the  APN-175  representing  the  oldest  technology. 

3.2.7  Program  Deletions 

It  had  originally  been  the  intent  to  incorporate  Redeye,  TOW, 
Shillelagh,  Improved  Hawk,  and  Sparrow  missiles  to  the  data  base  in  an 
effort  to  enrich  the  sample.  Early  in  the  data  compilation  effort,  how- 
ever, it  was  learned  that  RDT&E  information  for  the  Redeye,  TOW,  and 
Shillelagh  systems  was  sparse  in  both  cost  and  quantity  detail.  For  ex- 
ample, a visit  to  the  TOW  Project  Office  indicated  that  although  very 
aggregate  or  "bottom  line"  RDT&E  cost  data  were  available,  it  consisted 
of  cost  elements  from  the  inception  of  the  RDT&E  phase  through  its  com- 
pletion. At  best,  it  would  have  been  very  difficult  to  break  out  or 
even  estimate  prototype  production  costs.  With  respect  to  the  Redeye 
and  Shillelagh  RDT&E  costs,  information  was  virtually  unavailable. 

The  Improved  Hawk  and  Sparrow  missiles  were  excluded  from  the  data 
base  for  other  reasons.  In  the  case  of  the  Improved  Hawk,  it  could  not 
accurately  be  determined  what  elements  made  up  the  guidance  and  control 
subsystems  for  the  second,  third,  and  fourth  full-scale  production  con- 
tracts. The  reason  for  this  is  that  in  the  middle  of  the  second  produc- 
tion buy,  Raytheon  changed  accounting  systems.  This  necessitated  an 
amendment  to  their  governmental  cost  reporting  requirements.  In  the 
course  of  this  action,  the  WBS  structure  was  altered,  making  homogeneous 
comparisons  between  contracts  infeasible  within  the  time  frame  of  the 
study.  Further,  DD- 1737  quantity  information  for  the  last  three  produc- 
tion buys  is  no  longer  required  by  the  government  on  a routine  basis.  As 
such,  the  information  would  have  to  be  acquired  directly  from  Raytheon 
(Andover). 
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It  was  felt  that  the  Sparrow  missile  system  should  not  be  included  in 
the  study's  data  base  due  principally  to  the  large  degree  of  uncertainty  In 
the  costs  for  the  full-scale  production  phase.  Apparently,  once  full-scale 
production  began  and  after  a number  of  units  had  been  manufactured,  relia- 
bility and  performance  requirements  (MILSPEC)  were  made  more  stringent. 

This  caused  the  prices  for  the  latter  articles  to  be  somewhat  higher  in  cost 
than  the  earlier  models.  The  problem  was  determining  the  magnitude  of  this 
MILSPEC  change  on  costs,  since  to  incorporate  the  cost  data  without  an  ac- 
counting of  the  cost  Increase  would  lead  to  biased  and  erroneous  results 
when  fitting  the  cost  improvement  relationship.  This,  In  turn,  would  In- 
correctly cause  full-scale  production  costs  to  appear  greater  than  prototype 
costs. 
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4.0  HYPOTHESIS  TESTING 


The  three  dimensional  figure  of  Figure  4.1  reveals  the  breadth  of  the 
analysis  undertaken  in  this  study.  The  major  issue  to  be  addressed  by  the 
analysis  are  depicted  in  the  three  dimensions  of  the  figure.  The  checkmarks 
in  the  individual  blocks  indicate  the  areas  addressed  in  the  analysis.  We 
did  not  apply  log  linear  unit  theory  In  the  testing  of  the  sequential  hypoth- 
esis. This  area  was  deleted  because  we  did  not  expect  to  see  any  significant 
differences  between  the  sequential  and  disjoint  results  employing  unit  theory. 
Table  4.1  lists  the  algebraic  forms  of  the  models  that  were  statistically 

tested.  The  logarithmic  form  was  employed  because  log  normal  disturbances  arc 

* 

considered  most  appropriate.  Tjp  was  incorporated  In  the  dependent  variable 
In  order  to  constrain  the  exponent  on  Tjp  to  a value  of  one,  i.e.,  a propor- 
tional relationship  between  Tj  and  TJp  . Also  note  that  Models  5 through  8 
Involve  the  substitution  of  Cp  for  Tjp  In  Models  1 through  4,  respectively. 

The  results  of  the  hypothesis  testing  corresponding  to  the  blocks  of 
Figure  4.1  are  presented  in  Tables  4.2  through  4.4  . The  tables  contain  the 
values  of  the  coefficients  for  the  various  Independent  variables  and  fit 
statistics  for  the  various  models.  Each  of  the  issues  addressed  by  this 
study  will  be  considered  In  the  light  of  the  statistical  results  of  these 
tables. 

The  most  important  conclusion  that  can  be  made  based  on  the  results 

shown  in  the  tables  is  that  the  estimated  first  unit  prototype  cost  (T lp ) 

Is  a good  estimator  of  the  first  unit  production  cost.  Both  quantity  hypoth- 

2 

eses  involving  only  Tjp  produce  R in  the  .90  area  with  standard  errors 
under  0.3.  The  goodness  of  the  fit  is  better  observed  graphically  as  shown 
in  Figure  4.2  through  4.4  for  the  various  formulations  of  the  proportional 
model. 

* ~ 

This  specification  implies  a constant  percentage  error  across  the  range 
of  Tip  values.  It  also  prohibits  negative  values  of  the  ratio. 

2 

The  values  of  R were  calculated  on  variations  In  tnT^  , not  tnTj/Tlp 

The  latter  form  of  Independent  variable  was  necessary  to  constrain  the 
relationship  to  be  a proportional  one.  The  variation  in  T,  is  the 
proper  criterion,  I.e.,  the  matter  of  concern. 
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QUANTITY  THEORY 


FIGURE  4.1  SPECTRUM  OF  TESTED  MODELS 


Model  No. 1 In  Tj/Tjp  « InA 

2 In  Tj/Tjp  * InA  + aln  TjP 

3 In  Tj/TjP  * InA  + gin  Qp 

4 In  Tj/TjP  = InA  = a In  Tjp  + e In  Qp 

5-8  SUBSTITUTE  Cp  FOR  Tip  IN  EACH  OF  ABOVE  MODELS  (£p  NOT  ANALYZED 

IN  DISCONTINUOUS  CASE) 

TABLE  4.1  ALGEBRAIC  FORMS  OF  TESTED  MODELS 


TABLE  4.2 

HYPOTHESIS  TESTING  RESULTS 
CUM.  AVE.  THEORY 
SEQUENTIAL  HYPOTHESIS 


_2* 

MODEL  NO. 

InA 

lnTlP 

^Qp 

<5~ 

R 

t 1 

-.211 

.272 

.965 

1 2 

-.577 

.026X 

.279 

.965 

ALL  , 

1 J 

-.166 

- 

. 016X 

.281 

.965 

| 4 

-.676 

.042X 

. 040X 

.286 

.966 

t 1 

-.252 

.329 

.942 

1 

MISSILES  2 

-3.654 

.244Y 

.225 

.980 

1 3 

-2.007 

.464X 

.279 

.969 

\ 4 

-5.253 

-.561  X 

. 750X 

.233 

.985 

t 1 

-.192 

.258 

.972 

OTHER  2 

.012 

-.015X 

.270 

.973 

ELECTRONICS  , 

1 3 

-.904 

- 

.044X 

.268 

.973 

i 4 

-.089 

-.0004X 

.044X 

.284 

.973 

‘CALCULATED  WITH  lnTx 

AS  DEPENDENT 

VARIABLE: 

not 

lnTl/TlPl 

SIGNIFICANCE  LEVELS 

x = >10* 

Y * > 5< 

Z = >2.5* 


TABLE  4.3 


HYPOTHESIS  TESTING  RESULTS 
CUM.  AVE.  COST  THEORY 
DISJOINT  HYPOTHESIS 


MODEL  NO.  InA 

1 nTlP 

InQp 

InCp 

or ' 

R2* 

i 

1 

-.575 

.294 

.884 

• 

2 

' 1.403 

-.148Z 

• 

.266 

.911 

3 

-.281 

* 

— . 1 09Y 

.277 

.904 

a; 

.L 

4 

1.206 

—.11 8Y 

-.073X 

.264 

.919 

5 

..130 

.453 

.620 

6 

1.187- 

-.083X 

.464 

.628 

7 

-.508 

.236 

.382 

.747 

r 

8 

.320 

..233 

-.065X 

.392 

.752 

i 

1 

-.519 

.262 

.919 

2 

1.769 

-.171 

.193 

.961 

3 

-.353 

-.075X 

.265 

.926 

OTHER 

ELECTRONICS 

4 

1.752 

-.169 

-.009X 

.204 

.961 

5 

-.0295 

.292 

.842 

6 

.252 

-.022X 

.307 

.843 

7 

-.432 

. 182Z 

.246 

.899 

8 

.210 

.188Y 

-.051 

.256 

.903 

1 

-.698 

.354 

...  .811 

i 

MISSILES 

2 

-1.319 

.046X 

.407 

.812 

3 

.803 

-.397X 

.344 

.866 

4 

-4.794 

.599Z 

-1.061Z 

.170 

.978 

* 

Calculated  with  InTjas  dependent  variables;  not  1 r*T1/T1p 


SIGNIFICANCE  LEVELS 
X * >10S 
Y * > 5% 

Z = >2.5% 
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TABLE  4.4 


HYPOTHESIS  TESTING  RESULTS 
LOG-LINEAR  UNIT  THEORY 
DISJOINT  HYPOTHESIS 


MODEL 

NO.  InA 

lnTlp 

InQp 

lnCp 

XT 

1 

-.658 

.287 

2 

..598 

-•095X 

.284 

3 

-.484 

-.064X 

.288 

A 

.1 

4 

.521 

-.079X 

-.047X 

.289 

5 

-.115 

.360 

6 

-.0083 

-.0084X 

.373 

7 

-.650 

.198 

.295 

8 

-.745 

.198 

.007X 

.306 

1 

-.625 

.282 

2 

.826 

-.110X 

.271 

3 

-.552 

-.033X 

.295 

OTHER 

.840 

—.1 1 2X 

ELECTRONICS 

4 

.008X 

.288 

5 

-.224 

.271 

6 

-.723 

.039X 

.284 

7 

-.657 

.196 

.205 

i 

8 

-.766 

. 1 95Z 

.009X 

.218 

SIGNIFICANCE  LEVELS 
X - >10* 

Y ■ > 5* 

Z - >2.5* 
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SEQUENTIAL  MODEL,  LOG  LINEAR  CUM.  AVE.  THEORY 


•Till 


FIGURE  4.4  DISJOINT  MODEL,  LOG  LINEAR  UNIT  THEORY 


We  do  not  note  significant  differences  in  any  of  the  hypotheses  across 
the  two  broad  hardware  categories.  Although  the  missile  data  produces  poorer 
fits  in  all  the  formulations,  they  do  produce  factors  (mA)  that  are  very 
similar  to  those  estimated  for  the  other  electronics  group.  The  estimated 
value  of  the  factor  (Model  No.  1)  for  other  electronics  is  well  within  the 
probability  distribution  produced  by  the  missile  data.  Because  we  cannot 
distinguish  with  a high  degree  of  confidence  the  difference  between  the 
factors  in  the  proportional  model  under  either  quantity,  hypothesis  leads 
us  to  conclude  that  separate  models  should  not  be  employed  for  different 
hardware  and  that  the  aggregate  factors  should  be  applied. 


I 


Comparing  the  results  of  the  sequential  and  disjoint  hypothesis  re- 
veals little  difference  in  the  fit  characteristics  of  either  theory.  The 

2 

sequential  theory  produces  a better  R in  the  factor  only  model  (Model 
Ho.  1);  however,  the  standard  errors  are  virtually  the  same  (.294  vs.  .272). 
A notable  characteristic  of  the  disjoint  theory  is  the  significance  of  the 
other  two  variables  included  in  Model  No.  2 and  3.  A possible  interpreta- 
tion of  the  coefficient  on  prototype  quantity  which  produces  a smaller  Tj 
for  larger  prototype  quantities  is  that  the  sequential  theory  is  more  appro- 
priate. In  the  sequential  theory,  the  cost  of  the  first  production  unit 
(Unit  Number  Qp+1)  decreases  as  Qp  increases  because  of  the  negative 
slope  of  the  cost  quantity  relationship. 

The  meaning  of  the  significance  of  the  Tjp  term  of  Model  No.  2 
under  the  disjoint  theory  is  unclear.  The  coefficient  produces  a smaller 
factor  for  larger  cost  subsystems.  Its  magnitude,  however,  is  not  large 
enough  to  make  its  impact  clearly  visible  in  Figure  4.3.  This  phenomenon 
is  not  observed  in  the  sequential  theory  except  in  the  missile  group  where 
It  is  opposite  in  sign.  The  difficulty  of  interpreting  its  meaning  and  the 
inconsistency  between  quantity  theory  impacts  leads  us  to  exclude  the 
variable  from  selected  models. 
t 

We  are  able  to  conclude  that  the  factor  A is  not  equal  to  one  which  implies 
no  break  or  discontinuity  In  the  cost-quantity  relationship  under  the  se- 
quential theory.  The  probability  that  A is  less  than  one  is  .995. 
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The  statistical  evidence  seems  to  support  the  sequential  theory, 
although  only  to  a slight  degree.  A major  concern  of  ours  in  the  appli- 
cation of  this  theory  is  the  corresponding  improvement  rate  increase  ob- 
served in  the  production  data.  The  averaqe  production  improvement  rate 
for  the  disjoint  theory  and  the  total  sample  was  85. 2%.  Wheq  the  prototype 
quantity  is  treated  as  a missing  lot  in  the  sequential  theory,  the  improvement 
rate  steepens  to  an  average  of  78.7%.  This  is  a rate  which  is  much  lower 
than  what  is  typically  accepted  by  cost  analysts.  This  steepening  results 
from  the  fact  that  addition  of  Qp  to  production  lot  quantities  moves 

small  quantity  points  farther  to  the  right  than  large  quantities  in  log 

* 

space  on  cost-quantity  graph. 

Our  conclusion  based  on  the  above  observations  on  the  quantity 
theories  is  that  we  cannot  reject  either  hypothesis  on  statistical  grounds 
in  that  each  produces  a very  good  factor-only  model.  Our  recommendation 
is  to  apply  either  one  with  the  appropriate  factor  and  with  the  understand- 
ing that  the  sequential  model  implies  a higher  improvement  rate. 

Investigation  of  the  fit  statistics  resulting  from  using  different 
cost  improvement  theories  indicate  little  difference  for  either  theory  in 
the  disjoint  formulation.  There  is  as  suggested  in  Section  2.0  a differ- 
ence between  the  factors  produced  (.563  vs.  .518)  and  the  unit  theory  factor 
is  smaller  as  hypothesized.  We  cannot,  however,  establish  whether  this  is 
the  result  of  positive  bias  in  the  cumulative  average  theory  or  negative 

bias  In  the  unit  theory.  We,  therefore,  cannot  conclude  that  one  theory 

** 

is  better  than  the  other.  Our  recommendation  is  to  apply  either  with  the 
corresponding  factor  developed  in  the  analysis. 


* 

The  improvement  rate  of  90%  estimated  for  the  doppler  navigation  systems 
was  employed  in  both  quantity  theory  determinations. 

** 


It  is  possible  that  one  theory  applies  in  some  cases  and  not  in  others. 


The  remaining  issue  is  whether  Tjp  obtained  by  applying  observed 
production  improvement  rates  to  prototype  average  unit  costs  is  a better 
estimator  than  the  average  prototype  cost,  Cp  . This  is  clearly  the  case 
where  only  the  Cp  term  is  employed.  Tables  4.3  and  4.4  reveal  much  higher 
standard  errors  for  this  case  (.45  vs.  .29  in  cumulative  average  theory). 

The  incorporation  of  a quantity  term  along  with  Cp  , however,  improves  the 
fits  substantially--almost  to  the  point  that  the  fits  are  indistinguishable. 
The  exponent  on  the  prototype  quantity  term  corresponds  almost  exactly  to 
the  average  improvement  rate  observed  in  the  production  data.  This  exponent 
exhibits  a high  t-value  and  supports  the  hypothesis  that  there  is  cost  im- 
provement in  the  production  of  prototypes  and  that  the  rate  is  not  signifi- 
cantly different  from  the  production  rate.  Note  also  in  Table  4.4  the 
essentially  equivalent  inA  terms  of  Models  1 and  7 (-.650  vs.  .658)  when  a 
Qp  variable  is  added.  Given  an  average  improvement  rate,  these  two  models 
produce  the  equivalent  result.  Because  the  cumulative  average  theory  results 
are  better  if  Tjp  is  calculated  using  production  improvement  rate,  we  sug- 
gest that  the  Tjp  model  be  employed  where  an  improvement  different  than 
the  average  is  postulated.  Otherwise,  either  Model  1 r Model  7 give  essen- 
tially the  same  result. 

Our  recommendations  on  model  usage  are  summarized  in  Table  4.5  for 
each  quantity  and  improvement  theory. 
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IMPROVEMENT  THEORY 


TABLE  4.5 

RECOMMENDATIONS  ON  MODEL  USAGE 


LOG-LINEAR 

CUM.  AVG. 

SEQUENTIAL* 

Tj  “ .810Tlp 

DISJOINT 

* 

Tj  = .563Tlp 
or  Tj  = . 602Cp  Qp’236 

LOG-LINEAR 

UNIT  THEORY 

SAME  AS  ABOVE 

Tj  = .5l8Tlp 
or  Tj  = . 522Cp  Qp‘198 

CpQpbi  where  = Programs  Production  Improvement  Rate 
Theoretical  First  Unit  Cost 
Average  Prototype  Cost 
Prototype  Quantity 


The  application  of  sequential  theory  implies  higher 
improvement  rates  than  in  disjoint  theory. 
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APPENDIX  A 

ITERATIVE  PROCEDURES 
FOR  FITTING  COST  IMPROVEMENT  CURVES 
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* 

ITERATIVE  PROCEDURES  FOR  FITTING  COST  IMPROVEMENT  CURVES 

1 . INTRODUCTION  AND  BACKGROUND  * 

(U)  Compilation  and  analysis  of  cost  data  consumes  a substantial 
portion  of  the  cost  analyst's  resources  in  most  study  efforts.  While 
there  are  no  short  cuts  for  locating  and  acquiring  useful,  detailed 
cost  data,  it  would  be  beneficial  if  maximum  use  can  be  made  of  data 
once  it  is  in  hand  and  if  the  routine  and  repetitious  procedures  of 
normalization  for  fiscal  escalation  and  cost  improvement  effects  were 
reduced  to  a mechanical  process  performed  by  a computer  program.  The 
impetus  for  this  paper  was  exactly  that  situation — a study  which  re- 
quired the  processing  of  substantial  quantities  of  historical  cost 
information  for  production  hardware. 

(U)  Maximum  use  of  data  has,  on  occasion,  been  frustrated  by  the 
fragmentary  nature  of  the  information  available  for  some  production 
programs.  Cost  and  quantity  data  may  be  available  for  the  second, 
sixth,  seventh,  and  ninth  production  lots,  for  example;  but  the 
development  of  log-linear  cumulative  average  cost  improvement  curves 
directly  from  this  partial  information  is  not  possible.  Consequently, 
considerable  effort  (and  money)  may  be  spent  in  backfilling  or  trying 
to  "dig-out"  the  complete  program  history,  often  without  success.  In 
the  past,  the  only  alternative  was  to  modify  one's  approach  and  adopt 
log-linear  Individual  unit  cost  improvement  curve  theory  and  apply 
established  iterative  methods  to  quantitatively  define  the  cost  improve- 
ment curve.  It  is  not  clear  which  of  these  theories,  if  either,  more 
nearly  approximates  the  real  world  of  the  U.S.  defense  industry. 


This  Appendix  is  extracted  from  a paper  of  the  same  title, 
Tecolote  Research,  Inc.,  TM-28,  by  A.  J.  Kluge,  dated  July  1975. 
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(11)  The  purpose  In  this  paper  is  not  to  validate  one  or  the  other 
of  these  theories.  Rather,  it  is  to  show  that  the  same  iterative 
procedure  may  be  applied  to  either  theory  and  to  describe  a computer 
program  for  Implementing  the  methods.  Results  of  these  procedures 
will  also  be  presented  and  compared.  Perhaps  the  greatest  benefit 
of  this  computer  program  will  be  the  ability  to  accept  and  process 
directly  information  reported  in  the  Cost  Information  Reports  (CIR), 
the  DD-1177  Forms,  Contractor  Cost  Data  Reports  (CCDR)  and  other 
official  cost  reporting  systems. 

2.  MATHEMATICAL  FORMULATION 

(U)  The  problem  is  one  of  determining  values  for  the  parameters  of 
an  equation  which  describes  the  cost  improvement  curve,  given  lot  cost 
(either  total  cost  or  lot  average  unit  cost)  and  lot  production  unit 
numbers  for  two  or  more  production  lots.  If  all  of  the  lots  are 
sequential,  and  the  first  lot  begins  with  (production)  unit  number  1, 
the  problem  is  simply  one  of  accumulating  consecutive  lot  total  costs, 
accumulating  consecutive  lot  total  production  quantities,  dividing  one 
by  the  other  and  calculating  the  cumulative  average  unit  cost  curve. 
If,  however,  the  cost  information  for  the  first  lot  is  not  available, 
this  direct  approach  is  not  possible.  In  general,  if  information  for 
any  lot  is  missing,  the  information  for  subsequent  lots  cannot  be  used 
in  this  direct  method. 

2.1  DIRECT  APPROACH 

(U)  What  is  termed  the  direct  approach  in  this  paper  is  the  standard 
method  of  developing  cumulative  average  data  from  lot  or  contract 
information. 
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(U)  Given  the  total  cost  for  several  consecutive  lots  (or  contracts), 
the  cumulative  average  cost  C versus  cumulative  quantity  is  obtained  as 
follows; 


» 


C 


T1  T2 

Ql+Q2 


Q1+Q2^3 


T3 


Sl+V- * ‘ *^CTi 
Q j ^2"^  • • • • 


(U)  The  cumulative  average  costs  are  thus  determined  directly  and  the 
resulting  cost  versus  quantity  data  may  be  fit  with  the  log-linear 
improvement  curve  equation.  Regression  methods  may  be  used  in  this 
fitting  process. 


(U)  If  the  datum  C^j  were  not  available,  it  would  not  be  possible  to 
evaluate  Cj  or  any  of  the  other  C^'s.  An  iterative  procedure  using  lot 
average  unit  costs  may  be  used  instead.  This  procedure  is  described  in 
Sections  2.2  and  2.3. 


2.2  LOG-LINEAR  CUMULATIVE  AVERAGE  COST  THEORY 

(U)  To  make  use  of  lot  average  unit  cost  data  in  constructing  log-linear 
cumulative  average  cost  curves,  begin  with  the  following  definitions: 


where 


c - C0Qb  (1, 

C is  the  cumulative  average  cost  from  unit  one 
through  any  quantity,  Q, 

Cq  is  the  unit  cost  of  the  first  production  article. 
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Q is  the  cumulative  quantity, 
b is  the  improvement  curve  exponent  given  by 


b - log2S 


logNS 
logN2  # 


S is  the  improvement  curve  slope  expressed  as  a 
fraction  (e.g.  90%  = 0.90), 

Then  the  lot  average  unit  cost,  C^,  is  given  by: 

* Co«L1+b  - W»1+b 


Ql  - (Qf-I) 


or 


(2) 


where  Q is  the  unit  number  of  the  last  unit  in  the  lot,  and 

JL 

Qj,  is  the  unit  number  of  the  first  unit  in  the  lot. 

(U)  The  unit  numbers  and  total  lot  costs  (C^  x lot  quantity)  are 
usually  available  for  two  or  more  lots.  Our  goal  is  to  find  Cq  and  b. 
The  approach  is  to  find  quantity  plot  points  for  the  given  values, 
such  that  the  data  points  will  fall  on  the  log-linear  cumulative  average 
cost  improvement  curve.  Referring  to  Figure  (1),  we  are  given  C^(2), 
C^(3),  C^(4),  and  wish  to  find  the  quantity  plot  points  Q(2),  Q(3),  and 
Q(4) , where  the  values  equal  the  values  of  the  log-linear  cumulative 
average  cost  curve.  (For  purposes  of  illustration,  it  is  assumed  that 
Lot  1 cost  data  is  not  available.) 


(U)  The  problem  is  then  one  of  finding  values  for  Cq  and  b which 
satisfy  the  requirement: 


Unclassified 


or 


or  to  find  Q where:  Q 


<Qf-D 

<Qf-» 


l+b 


1/b 


(3) 


(U)  Since  b is  unknown,  equation  (3)  does  not  lend  itself  to  direct 
solution,  nor  will  it  yield  to  solution  by  simultaneous  equations.  But, 
an  iterative  procedure  is  possible  as  follows. 

(1)  Estimate  initial  quantity  plot  points  for  each  lot; 
e.g.,  the  arithmetic  mid-point  of  each  lot; 

(2)  Use  these  initial  estimates  to  perform  a curve  fit  of  C 
versus  Q to  obtain  an  equation  of  the  form  C ■ CqQ  . 

(3)  Use  the  value  of  b resulting  from  the  curve  fitting 
procedure  to  refine  the  initial  estimates  of  Q by 
substituting  b in  equation  (3). 

(4)  The  refined  estimates  of  Q are  used  to  refit  the  curve 

C - C Qb. 
o 

(5)  Repeat  Steps  3 and  4 above  until  the  quantity  values 
converge. 

(U)  The  resulting  equation  provides  the  sought  for  parameters  Cq  and 
b for  the  log-linear  cumulative  average  cost  improvement  curve. 

2.3  LOG-LINEAR  INDIVIDUAL  UNIT  COST  THEORY 

(U)  If  log-linear  unit  cost  theory  is  adopted,  the  individual  unit 
cost,  C,  is  given  by  the  equation 

C - CoQb  (4) 
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■ 


where  CQ%  Qt  and  b are  as  previously  defined.  Then  the  cumulative 
average  cost  of  M units  is 


M 

& v*!1 

M 


(5) 


Because  this  equation  is  cumbersome 
approximated  by  its  asymptote,  that 


C 


1+b 


to  evaluate,  it  is  normally 
is; 


(6) 


It  is  then  possible  to  say  that; 


QLI+b  - «F-»1+b" 

oL  - <QF-l>  _ 

Employing  the  same  rationale  developed  earlier,  plot  points  are 
found  by  equating  C to  C . 

X 


or 


C 


o 

1+b 


(Qp-l)1+b 

<V1> 


QL1+l>  - <V1)1+b 


Ql  - (Qp-1) 


(7) 


(U)  Note  that  this  expression  differs  from  equation  (3)  by  the 
l/(l+b)  term  only.  It  is,  therefore,  possible  to  generalize  and  rewrite 
equations  (3)  and  (7)  as  follows: 


w I | v iw  V W « I ■ v 


where  Q is  the  plot  point  for  the  log-linear  cumulative  average 

cost  curve 

when  K * 0, 

or  Q ■ the  plot  point  for  the  log-linear  unit  cost  curve 

when  K » 1. 


t 


S 


3.  TYPICAL  RESULTS 


(U)  Samples  of  the  output  of  a computer  program,  using  the  three 
curve  fitting  methods  explained  above,  are  shown  on  the  following  pages. 
Figure  2 shows  the  results  of  a direct  curve  fit.  The  headiug  identifies 
the  data  used  and  the  date  on  which  the  program  was  run.  The  next  heading 
identifies  the  curve  fit  as  one  based  on  log-linear  cumulative  average 
cost  theory.  The  presence  of  only  one  entry  under  the  column  headed 
"Iteration  No."  indicates  that  the  direct  curve  fit  method  was  employed 
since  an  iterative  curve  fit  always  results  in  two  or  more  lines  of  output. 

(U)  Referring  to  the  equation  cost  ■ where  Q is  quantity,  the 

column  headed  "first  unit  cost"  gives  the  estimate  CQ  of  the  cost  of 
the  first  unit.  The  column  headed  "exponent"  gives  the  exponent  b and 
"slope"  gives  S - 2^.  "INDEX  OF  DETERMINATION"  is  a statistical  measure 
of  how  well  the  learning  curve  fits  the  data.  The  closer  the  index  of 
determination  is  to  1.0,  the  better  the  curve  fit.  The  columns  headed 


EPS1,  EPS2  do  not  apply  to  the  direct  curve  fit. 

(U)  Figure  3 is  a typical  page  of  output  cost  data  for  the  direct 
curve  fit.  For  convenience,  the  data  source  and  date  is  again  identi- 
fied by  the  first  printed  line.  The  first  line  of  data  gives  actual 


cumulative  average  costs  escalated  to  the  desired  fiscal  year  as 
indicated  by  the  heading,  "CUM.  AVG.  COST  IN  The  lines  printed 

under  "QUANTITY  PLOT  POINTS  AND  ESTIMATED  COSTS  IN  FY75  DOLLARS"  gives 
cumulative  quantities  and  directly  below  indicates  the  estimated  cost 
at  that  quantity  as  determined  by  the  curve  cost  ■ CoQ^.  This  page  of 
output  also  records  the  number  of  data  points  input  to  the  program, 
since  there,  is  one  column  of  figures  for  each  data  point.  The  entries 
allow  comparison  between  actual  costs  and  estimated  costs  on  a 


normalized  basis. 


n.irt'v-  Ifm-' 
U i ! C i l:  i’  O i .'  i 0 L* 
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(U)  Figure  4 of  the  sample  output  is  similar  to  Sample  1.  The  second 
heading  shows  that  a cumulative  average  cost  improvement  curve  is  being 
determined,  while  the  presence  of  more  than  one  iteration  indicates  that 
the  iterative  method  is  being  used.  The  line  labeled  "Iteration  No.  1" 
is  the  first  approximation  to  the  learning  curve,  line '2  is  the  second, 
etc.  While  the  index  of  determination  remains  a measure  of  the  goodness 
of  the  fit,  it  has  a somewhat  different  interpretation  when  one  of  the 
iterative  processes  is  employed.  For  this  reason,  caution  should  be 
exercised  if  any  comparison  is  made  between  the  indices  of  determination 
given  under  direct  and  iterative  curve  fits. 

(U)  The  next  page  affords  a comparison  of  the  input  data  with  the 
estimated  costs  based  on  the  curve  fit.  Under  the  heading,  "CUM.  AVG. 
COSTS..."  are  the  costs  based  on  the  input  data.  These  costs,  at  the 
quantities  indicated  for  each  iteration,  can  be  compared  with  the  costs 
estimated  by  the  curve  fit  at  the  same  quantity.  This  comparison  is 
listed  for  each  iteration. 

(U)  When  the  iterative  process  is  employed,  the  columns  labeled  EPS1 
and  EPS2  measure  the  convergence  of  the  method.  EPS1  is  the  fractional 
change  in  the  calculated  first  unit  cost  between  successive  iterations. 
EPS 2 is  the  similar  change  in  the  exponents.  When  EPS1  and  EPS 2 both 
become  small,  the  iteration  process  stops. 

(U)  Figures  6 and  7 of  the  sample  output  are  the  same  as  Figure  4 
and  5,  except  that  log  linear  unit  cost  theory  is  being  used  to  develop 
unit  cost  curves  instead  of  cumulative  average  cost  curves.  Column 
headings  on  Figure  6 have  the  same  meanings  as  those  on  Figure  4. 

(U)  Figure  7 again  gives  costs  based  on  the  input  data  for  comparison 
with  costs  estimated  by  the  learning  curve  at  the  quantities  indicated 
for  each  iteration. 
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FIGURE  3 
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APPENDIX  B 


THE  DEVELOPMENT  OF  A FULL  SAMPLE  RELATIONSHIP 


*•  • 

1.0  INTRODUCTION 
f 

In  the  main  body  of  this  paper,  models  were  produced  which  estimated 
the  production  first  unit  cost,  Tj  , as  a function  of  prototype  first  unit 
cost,  Tjp  . The  derivation  of  the  models  was  a two  step  process.  For  each 
program  in  the  data  base,  T^  was  estimated  by  the  regression  of  costs  at 
various  quantities  with  their  associated  quantities.  The  models  were  then 
derived  by  fitting  Tj  to  a function  of  T^p  . The  application  of  these 
models  involves  a similar  two  step  process.  Tj  is  estimated  as  a function 
of  Tjp  and  then  an  extrapolation  is  made  along  a cost  improvement  curve 
to  obtain  a cost  estimate  at  the  desired  quantity. 

This  appendix  discusses  an  alternate  approach  which  eliminates  the 
multi-step  process  by  incorporating  ail  of  the  production  cost-quantity 
data  points  in  the  model.  The  full  sample  is  then  used  to  estimate  the 
cost-quantity  relationship  so  that  cost  at  quantity  is  given  directly  as 
a function  of  prototype  cost  and  production  quantity. 
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2.0  MODEL  DESCRIPTION 

The  simplest  model  formulation  which  accounts  for  both  the  cost- 
quantity  relationship  and  the  prototype- full  scale  production  relation- 
ship Is  given  by 

COST  = A TlpQb 

In  this  formula,  COST  is  the  cumulative  average  (or  unit)  cost  of 
full  scale  production  at  quantity  Q . A describes  the  drop  in  cost 
relative  to  the  prototype  first  unit  cost  TJp  . The  exponent  b is 
the  implied  cost  improvement  curve  slope.  Notice  that  this  model  con- 
strains the  cost-quantity  relationship  to  a single  cost  Improvement  slope 
for  all  programs  and,  therefore,  constrains  the  analyst  using  this  model 
to  a single  slope.  If  it  is  believed  that  different  programs  will  ex- 
hibit different  cost  improvement  rates,  then  the  model  suffers  from 
specification  error. 


n /nb 


Since  Tjp  is  calculated  by  T^p  = Cp/Qp  , and  since  we  are 
assuming  a single  slope  for  all  programs,  the  above  model  can  be  written: 


COST 


aA  Qb 


or  COST  = A C r 


where 


'P  \ Qp  / 

the  parameters  to  be  found  by  regression  are  A and  b . 


In  order  to  investigate  the  effects  of  Cp  and  Qp  on  the  basic 
relationship  postulated  above,  additional  Cj,  and  Qp  terms  can  be 
added  to  the  model.  When  this  is  done,  the  four  basic  models  resemble 
the  four  basic  model  forms  given  In  the  main  body  of  this  paper.  These 
models  are  given  below  in  Table  1. 
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TABLE  1 
MODEL  FORMS 

FORM 

I 

COST  . A Ep  (2-)  b 

C0ST  = A (it) b v 

C0ST  ■ A (q^)  6 «P 

C0ST  ■ A (St) b V Op6 
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3.0  REGRESSION  ANALYSIS 


The  regression  analysis  was  performed  on  the  log  transforms  of  the 
models  given  in  Table  1.  With  the  inclusion  of  all  the  appropriate  cost 
and  quantity  information,  the  data  base  was  the  same  as  that  used  in  the 
main  study.  The  same  equipment  stratification  of  the  data  base  was  made 
and  both  major  hypotheses  were  investigated.  The  investigation  was  limited 
to  cumulative  average  cost  improvement  curve  theory  only. 

The  results  of  the  regression  analysis  are  given  in  Tables  2 and  3. 

In  Tables  2 and  3,  the  column  labeled  Model  indicates  the  model  being 
tested  and  the  sample  used  for  the  test.  The  column  labeled  tnA  gives 
intercept  term  for  the  model;  while  the  columns  labeled  Cp  and  Qp  give 
the  exponents  « and  6 of  the  independent  variables  Cp  and  Qp  . The 
column  labeled  SX  gives  the  implied  cost  improvement  curve  slope  in  per- 
cent. S is  obtained  as  100x2b  where  b is  the  exponent  of  Q/Qp  in 
the  various  models.  The  last  column  gives  the  standard  error  of  the 
estimate  for  each  of  the  models. 
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TABLE  2 

DISJOINT  HYPOTHESIS 


S% 


MODEL 

tnA 

co 

flj. 

S=100x2bf 

a — 

TOTAL 

N*63 

1 

-.642 

82.4 

.379 

2 

-.536 

-.008X 

82.4 

.382 

3 

-.048 

-.206 

82.9 

.317 

4 

.226 

.020X 

0 

CVJ 

• 

1 

82.6 

.320 

MISSILES 

N-27 

1 

-1.056 

85.1 

.300 

2 

-4.448 

-.433 

82.9 

.270 

3 

1.236 

-.531 

80.9 

.264 

4 

3.084 

-.202X 

-.366 

81.2 

.266 

ELECTRONICS 

N=36 

1 

-.671 

87.6 

.318 

2 

1.478 

-.159 

87.0 

.289 

3 

-.451 

1 

• 

o 

o> 

86.6 

.314 

4 

1.433 

-.153 

-.014X 

86.8 

.293 

MODEL:  COST  - A 6 TTp- Q» 

X Indicates  the  parameter  is  not  significant  at  the  10%  level. 
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TABLE  3 

SEQUENTIAL  HYPOTHESIS 


MODEL 

inA 

cp 

«P 

S*  r 

a — 

TOTAL 

N*63 

1 

-.394 

79.1 

.377 

2 

1.716 

-.149 

76.7 

.355 

3 

.141 

-.203 

80.1 

.319 

4 

2.116 

-.140 

-.198 

77.8 

.295 

MISSILES 

N=27 

1 

-.755 

82.2 

.324 

2 

4.701 

-.425 

79.6 

.299 

3 

1.303 

-.457 

78.0 

.302 

4 

3.836 

-.274 

-.236 

78.3 

.301 

ELECTRONICS 

Ns36 


1 

-.375 

80.1 

.398 

2 

3.127 

-.253 

78.9 

.307 

3 

.446 

-.242 

75.7 

.328 

4 

2.610 

-.133  -.183 

76.8 

.292 

» t 

, . b 

MODEL: 

COST  ■ A £p 

t) 
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4.0  CONCLUSIONS 


The  conclusion  to  be  drawn  from  Tables  2 and  3 essentially  parallel 
the  conclusions  given  in  the  main  body  of  this  paper.  In  summary,  both 
the  disjoint  and  sequential  hypotheses  reproduce  their  respective  data 
bases  equally  well  and,  hence,  it  is  impossible  to  establish  the  best 
hypothesis  on  the  basis  of  Tables  2 and  3.  The  small  sample  size  in 
conjunction  with  the  similar  fits  obtained  in  the  missile  and  electronics 
samples  makes  it  Inadvisable  to  conclude  that  missiles  behave  inherently 
different  from  the  other  electronics  in  our  data  base.  Although  the 
addition  of  the  variables  Cp  and  Qp  generally  improve  the  fits,  the 
improvement  is  not  large  enough  to  conclude  with  confidence  that  the 
prototype/full  scale  relation  is  dependent  on  either  Cp  or  Qp  . 
Finally,  notice  that,  as  expected,  the  sequential  hypothesis  implies  a 
steeper  cost  improvement  curve  slope  in  conjunction  with  a smaller  drop 
from  prototype  production  costs. 

Based  on  the  above  observations,  the  two  best  models  are  given 

below. 

DISJOINT  HYPOTHESIS: 

CUMULATIVE  AVERAGE  COST 


SEQUENTIAL  HYPOTHESIS: 

CUMULATIVE  AVERAGE  COST  * .67  Cp(  jj-)  *34 


